Controlled Experimentation in Naturalistic Mobile Settings

Performing controlled user experiments on small devices in naturalistic mobile settings has always proved to be a difficult undertaking for many Human Factors researchers. Difficulties exist, not least, because mimicking natural small device usage suffers from a lack of unobtrusive data to guide experimental design, and then validate that the experiment is proceeding naturally.Here we use observational data to derive a set of protocols and a simple checklist of validations which can be built into the design of any controlled experiment focused on the user interface of a small device. These, have been used within a series of experimental designs to measure the utility and application of experimental software. The key-point is the validation checks -- based on the observed behaviour of 400 mobile users -- to ratify that a controlled experiment is being perceived as natural by the user. While the design of the experimental route which the user follows is a major factor in the experimental setup, without check validations based on unobtrusive observed data there can be no certainty that an experiment designed to be natural is actually progressing as the design implies.Comment: 12 pages, 3 table

Patterns in Eyetracking Scanpaths and the Affecting Factors

Web pages are typically decorated with different kinds of visual elements that help sighted people complete their tasks. Unfortunately, people accessing web pages in constrained environments, such as visually disabled and small screen device users, cannot benefit from them. In our previous work, we show that tracking the eye movements of sighted users provide good understanding of how people use these visual elements. We also show that reengineering web pages by using these visual elements can improve people's experience in constrainted environments. However, in order to reengineering web pages based on eyetracking, we first need to aggregate, analyse and understand how a group of people's eyetracking data can be combined to create a common scanpath (namely, eye movement sequence) in terms of visual elements. This paper presents an algorithm that aims to achieve this. This algorithm was developed iteratively and experimentally evaluated with an eyetracking study. This study shows that the proposed algorithm is able to identify patterns in eyetracking scanpaths and it can work well with different number of participants. We then extended our experiments to investigate the effects of the task, gender and familiarity factors on common scanpaths. The results suggest that these factors can cause some differences in common scanpaths. This study also suggests that this algorithm can be improved by considering different techniques for preprocessing the data, by addressing the drawbacks of using the hierarchical structure and by taking into account the underlying cognitive processes

Less users more confidence: How AOIs don’t affect scanpath trend analysis

User studies are typically difficult, recruiting enough users is often problematic and each experiment takes a considerable amount of time to be completed. In these studies, eye tracking is increasingly used which often increases time, therefore, the lower the number of users required for these studies the better for making these kinds of studies more practical in terms of economics and time expended. The possibility of achieving almost the same results with fewer users has already been raised. Specifically, the possibility of achieving 75% similarity to the results of 65 users with 27 users for searching tasks and 34 users for browsing tasks has been observed in scanpath trend analysis which discovers the most commonly followed path on a particular web page in terms of its visual elements or areas of interest (AOIs). Different approaches are available to segment or divide web pages into their visual elements or AOIs. In this paper, we investigate whether the possibility raised by the previous work is restricted to a particular page segmentation approach by replicating the experiments with two other segmentation approaches. The results are consistent with ~5% difference for the searching tasks and ~10% difference for the browsing tasks

Eye tracking scanpath analysis techniques on web pages: A survey, evaluation and comparison

Eye tracking has commonly been used to investigate how users interact with web pages, with the goal of improving their usability. This article comprehensively revisits the techniques that could be applicable to eye tracking data for analysing user scanpaths on web pages. It also uses a third-party eye tracking study to compare these techniques. This allows researchers to recognise existing techniques for their goals, understand how they work and know their strengths and limitations so that they can make an efficient choice for their studies. These techniques can mainly be used for calculating similarities/dissimilarities between scanpaths, computing transition probabilities between web page elements, detecting patterns in scanpaths and identifying common scanpaths. The scanpath analysis techniques are classified into four groups by their goals so that researchers can directly focus on the appropriate techniques for a sequential analysis of user scanpaths on web pages. This article also suggests dealing with the limitations of these techniques by pre-processing eye tracking data, considering cognitive processing and addressing their reductionist approach

Group versus Individual Web Accessibility Evaluations: Effects with Novice Evaluators

We present an experiment comparing performance of 20 novice evaluators of accessibility carrying out Web Content Accessibility Guidelines 2.0 conformance reviews working individually to performance obtained when they work in teams of two. They were asked to first carry out an individual assessment of a web page. Later on, they were matched randomly to constitute a group of two and they were asked to revise their initial assessment and to produce a group assessment of the same page. Results indicate that significant differences were found for sensitivity (inversely related to false negatives: +8%) and agreement (when measured in terms of the majority view: +10%). Members of groups exhibited strong agreement on the evaluation results among them and with the group outcome. Other measures of validity and reliability are not significantly affected by group work. Practical implications of these findings are that, for example, when it is important to reduce the false-negative rate, then employing a group of two people is more useful than having individuals carrying out the assessment. Openings for future research include further explorations of whether similar results hold for groups larger than two or what is the effect of mixing people with different accessibility background. RESEARCH HIGHLIGHTS When novice accessibility evaluators work in groups, their ability to identify all the true problems increases (by 8%). Likewise, reliability of group evaluations increases (by 10%). Individual or group evaluations can be considered as equivalent methods with respect to false positives (if differences up to 8% in correctness are tolerated). Individual or group evaluations can be considered as equivalent methods with respect to overall effectiveness (if differences up to 11% in F-measure are tolerated)

Corporate Governance and the Cost of Debt of Large European Firms

This paper examines the effects of different corporate governance mechanisms on the cost of debt for large European firms and documents a novel interaction effect between shareholder rights and disclosure. Improved disclosure leads to a lower credit spread only if shareholder rights are low. A possible explanation for this finding is the ‘share rights or disclose’ hypothesis. If shareholders have sufficient rights to monitor and influence management decisions, debt providers can rely upon shareholders to mitigate agency costs. Otherwise, bondholders require a premium to compensate for the information risk due to uncertainty about the true value of the firm

Detecting high-functioning autism in adults using eye tracking and machine learning

This is an accepted manuscript of an article published by IEEE in IEEE Transactions on Neural Systems and Rehabilitation Engineering on 30/04/2020, available online: https://ieeexplore.ieee.org/document/9082703 The accepted version of the publication may differ from the final published version.The purpose of this study is to test whether visual processing differences between adults with and without highfunctioning autism captured through eye tracking can be used to detect autism. We record the eye movements of adult participants with and without autism while they look for information within web pages. We then use the recorded eye-tracking data to train machine learning classifiers to detect the condition. The data was collected as part of two separate studies involving a total of 71 unique participants (31 with autism and 40 control), which enabled the evaluation of the approach on two separate groups of participants, using different stimuli and tasks. We explore the effects of a number of gaze-based and other variables, showing that autism can be detected automatically with around 74% accuracy. These results confirm that eye-tracking data can be used for the automatic detection of high-functioning autism in adults and that visual processing differences between the two groups exist when processing web pages.Published versio

“Keep it simple!”: an eye-tracking study for exploring complexity and distinguishability of web pages for people with autism

A major limitation of the international well-known standard web accessibility guidelines for people with cognitive disabilities is that they have not been empirically evaluated by using relevant user groups. Instead, they aim to anticipate issues that may arise following the diagnostic criteria. In this paper, we address this problem by empirically evaluating two of the most popular guidelines related to the visual complexity of web pages and the distinguishability of web-page elements. We conducted a comparative eye-tracking study with 19 verbal and highly independent people with autism and 19 neurotypical people on eight web pages with varying levels of visual complexity and distinguishability, with synthesis and browsing tasks. Our results show that people with autism have a higher number of fixations and make more transitions with synthesis tasks. When we consider the number of elements which are not related to given tasks, our analysis shows that they look at more irrelevant elements while completing the synthesis task on visually complex pages or on pages whose elements are not easily distinguishable. To the best of our knowledge, this is the first empirical behavioural study which evaluates these guidelines by showing that the high visual complexity of pages or the low distinguishability of page elements causes non-equivalent experience for people with autism.Published onlin

Web users with autism: eye tracking evidence for differences

Anecdotal evidence suggests that people with autism may have different processing strategies when accessing the web. However, limited empirical evidence is available to support this. This paper presents an eye tracking study with 18 participants with high-functioning autism and 18 neurotypical participants to investigate the similarities and differences between these two groups in terms of how they search for information within web pages. According to our analysis, people with autism are likely to be less successful in completing their searching tasks. They also have a tendency to look at more elements on web pages and make more transitions between the elements in comparison to neurotypical people. In addition, they tend to make shorter but more frequent fixations on elements which are not directly related to a given search task. Therefore, this paper presents the first empirical study to investigate how people with autism differ from neurotypical people when they search for information within web pages based on an in-depth statistical analysis of their gaze patterns